Communication-Efficient and Federated Multi-Agent Reinforcement Learning

In this paper, we consider a distributed reinforcement learning setting where agents are communicating with central entity in shared environment to maximize global reward. A main challenge is that the randomness of wireless channel perturbs each agent’s model update while multiple agents’ updates may cause interference when under limited bandwidth. To address issue, propose novel algorithm based on alternating direction method multipliers (ADMM) and “over air aggregation” using analog transmission scheme, referred as A-RLADMM. Our incorporates into formulation ADMM method, which enables transmit element their updated models over same communication. Numerical experiments multi-agent collaborative navigation task show our proposed significantly outperforms digital communication baseline A-RLADMM (D-RLADMM), lazily aggregated policy gradient (RL-LAPG), well versions vanilla FL, (A-FRL) (D-FRL) respectively.